Automatic Maintenance of Web Directories by Mining Web Browsing Data

Received December 24, 2010 Revised May 26, 2011 Web directories allow Web users to browse a hierarchy of categories, under which different types of resources are classified. We study the problem of maintaining a Web directory, that is, the problem of continually discovering and ranking resources that are relevant to the categories of the directory. We propose an unsupervised computational method that conducts the maintenance of the directory by analyses of user browsing data. The method is based on the extraction and classification of user sessions (sequences of resources selected by users) into the categories of the directory. In addition, we show that the directory maintenance method can be slightly modified to find queries that are useful to find relevant resources allowing users to switch from directory browsing to query formulation. Experimental results allow for affirmation that the proposed methods are effective, that they attain identification of new pages in each category and also recommend related queries with high precision, without needing labeled data to conduct traditional web page and query classification tasks.